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14.  ABSTRACT 

The  mission  of  the  Cognitive  Technology  for  Teams  (CTT)  research  program  is  to  conduct  research  and 
development  that  enhances  the  Air  Forces  capability  to  support  teams  that  are  effective,  resilient,  and 
adaptable  within  the  context  of  command  and  control  (C2).  The  program  provides  science  and  technology 
leadership  in  two  areas,  a)  the  development  and  assessment  of  collaborative  interfaces  to  extend  the 
effectiveness  of  battle  managers  working  within  a  network-centric  framework,  and  b)  the  development  of 
new  metrics  for  assessing  team  workload  and  performance.  To  this  end  the  program  conducts  applied 
research  within  two  laboratories.  The  Collaborative  Technology  Testbed  permits  the  systematic  evaluation 
of  advanced  collaboration  interface  technologies,  data  visualization  tools,  and  multi-modal  interface 
technologies  and  their  effects  on  team  performance,  communication  effectiveness,  shared  situation 
awareness,  and  decision  effectiveness.  Experiments  in  this  lab  typically  employ  high-fidelity  simulated 
work  environments  for  human-in-the-loop  experimentation.  The  Augmented  Team  Workload  Assessment 
Lab  is  designed  to  explore  the  application  of  physiologic-based  operator  state  assessment  technology  to  the 
objective,  online  measurement  of  team  states  such  as  mental  workload,  stress,  and  fatigue.  Research  in  this 
lab  is  focused  on  the  development  and  validation  of  theory-driven,  innovative  subjective  and  behavioral 
metrics  for  characterizing  individual  and  team  workload;  and  development  of  robust  physiological  indices 
of  team  workload,  with  a  particular  interest  in  minimally  invasive  measures  such  as  EEG,  EOG,  ECG  eye 
movement  data  and  cerebral  hemodynamics.  Current  research  directions  for  the  CTT  program  will  be 
discussed. 
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Warfighter  Interface  Division 


Battlespace  Acoustics  Branch  (RHCB) 

Leading  the  discovery,  evaluation,  and  transition  of  revolutionary 
auditory  and  communication  technologies  that  optimize  warfighter 
survivability  and  lethality  across  the  full  range  of  battlespace 
environments 

Supervisory  Control  Interfaces  Branch  (RHCI) 

Conducting  research  to  enhance  the  effectiveness  of  the  integration  of 
crew  and/or  operators  with  intelligent  and  autonomous  systems  to 
fully  exploit  the  joint  capabilities  of  the  human-machine  system. 

Collaborative  Interfaces  Branch  (RHCP) 

Leading  the  discovery  of  innovative  technologies  that  optimize  human- 
to-human  and  human-to-machine  collaboration  in  a  network-centric, 
distributed  environment  for  both  teams  and  individuals  across  all  USAF 
domains 

Battlespace  Visualization  Branch  (RHCV) 

Advancing  the  science  and  technology  associated  with  the  collection, 
optimization,  display,  and  assimilation  of  visually  complex  information 

to  enable  accurate  and  effective  decision  making  across  the  battlespace 
domains 
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Collaborative  Interfaces  for  C2 

Program  Goals 


Collaborative  Tools  for  Tactical  C2 
(FY  05-09) 

-  Design  Tools  -  with  multi-modal 
collaborative  interface  technologies 

-  To  Enhance:  Performance  Efficiency, 
Decision-Making,  Situation  Awareness, 
Workload 


Augmented  Team  Workload  Assessment 
(FY08-12) 

-  Develop  Metrics  to  Assess:  Team 
Cognitive  Workload  and  Situation 
Awareness 


-  To  Enhance:  Distribution  of  Workload, 
Situation  Awareness,  Efficiency  & 
Effectiveness  of  Decision-Making, 
Speed  of  Command 

•  Other  Studies/Research  Areas 


DRAW  Use  Case  1 : 

Dynamic  Replanning 


Can  operators  quickly  communicate  information  changes  (e.g.  new  EoB  info  that 
changes  plan),  and  insure  safety  and  success? 


Experiment  Players 


•  C2: 


-  Tactical  -  E-3 

-  Operational  -  CAOC  (White  Force) 

TST  Strike  Package: 

-  Strikers  -  GR-4s 


-  SEAD  -  F-16CJs 


Experimental  Factors 


•  Joint  Warrior-based  scenario 

—  TST  and  intel  injects  via  WF 

•  US  &  UK  operator  participation 

-  1-5  Mar,  Farnborough,  UK 

•  Experimental  focus: 

-  Interflight  coordination  for  support 

-  ROE  &  SEAD  asset  dependencies 

-  Quickly  communicating  changes 

-DRAW tool  (US) 

—  Dynamic  mission  replanning 

-  Mission  management  tools  (UK) 
—  Resulting  TTPs 
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Chat  Communication  Study 


Chat  communication  study  to  assess  the  impact  chat 
communication  may  have  on  communication  processes, 
shared  understanding,  and  sensemaking  behaviors;  all  of 
which  impact  communication  and  coordination  effectiveness. 

Approach 

Three  major  components: 

1  .Operational  Chat  Survey 

2.  Live-fly  Observation  (if  permissible) 

3.  Empirical  (Lab)  Study 


Greater  insight  into  the  impact  of  chat  communication 
can  help: 

•  Enhance  implementation  to  maximize  strengths 
and  minimize  weaknesses 

•  Focus  future  training 

•  Make  smart  decisions  on  best  practice  of  deploying 
technology 


Projected  Research  Schedule  As  of:  11  Dec  09 

Month  Nov  Jan  Mar  May  July 
Dec  Feb  Apr  Jun 

Data  Interview  with  E-3  SME's 

Synthetic  Task  Environment  Scenario  Design, 

Development  &  Testing 


Live-fly  Observation 

Distribute  Survey 

Collect  experimental  data 

Data  Analysis  &  Write-up  &  Final  Report 


■ 


Benefits  to  the  Warfighter 


Description 
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Chat  Communication  Study  mm 


Three  Components  of  Study: 

1 .  Operational  Survey 

Objective:  understand  current  practices,  procedures, 
issues,  usage,  concerns,  and  operator  requirements. 

2.  Field  Observation  (contingent  on  opportunity) 

Objective:  understand  chat  usage  and 
difficulties  and  domain  challenges 

3 .  Experimental  Study 

Objective:  empirically  test  the  impact  chat  technology 
on  how  teams  of  operators  use  this  tool  to  solve 
problems,  coordinate,  and  communicate 


2x2  Experimental  Design 


Voice 

Keyboard 

Transient 

Voice  Only 

Chat  Only  (messages 
disappear) 

Permanent 

Voice  +  Archival 
Chat  Log 

Chat  +  Archival  Chat 
Log 

Communication  =  a  means  to 
solve  complex  problems 
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Team  Resource  And  Cognitive 
Effectiveness  (TRACE)  Monitor 


Objective:  Develop  near-real-time  behaviorally-  and 

Power  Spectral  Density 

neurophysiologically-based  measures  of  team  fitness 
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•  Leverage  advanced  mathematical  techniques  to  recognize 
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patterns  in  team  behavioral  and  physiologic  data 
associated  with  effective  or  impaired  team  performance 

-  Statistical  modeling  of  team  communication  and  behavior  using 
Hidden  Markov  Models  (HMMs) 

-  Applying  nonlinear  data  analytic  techniques  (recurrence  quantification 
analysis,  cross-recurrence  quantification,  fractal  analysis,  etc.)  to 
identify  chaotic,  emergent  patterns  in  team  communication  and 
physiologic  data 


Fractal  analysis  of  human  inter-beat  interval  data 


HMM  of  operator  UAV  control  (from  Boussemart,  Las 
Fargeas,  Cummings,  &  Roy,  2009) 


•  Apply  online,  neurophysiological  measures  to  diagnose 
likely  drivers  of  team  performance  impairments  (extreme 
workload,  inequitably  distributed  workload,  stress,  fatigue, 
etc.) 

-  Potentially  useful  measures  have  been  derived  from  EEG,  ECG,  eye- 
gaze  tracking,  and  cerebral  hemodynamics  and  oximetry 


Cross-recurrence  analysis  of  eye-gaze  data  10 


TRACE  Monitor 


Relevance 

•  Future  network-centric  CONOPs  require  rapidly  formed,  distributed  teams  for  missions  such  as 
time-sensitive-targeting  (e.g.,  Alberts  &  Hayes,  2003). 

•  Distributed  teams  may  not  have  the  opportunity  to  develop  shared  mental  models  that  support 
good  team  performance  &  SA  (Salas  et  al.,  1995) 

•  TRACE  will  allow  remote  mission  commanders  and  adaptive  aiding  tools  to  perceive  and 
anticipate  team  “mental”  fitness,  allowing  them  to  better  direct  team  resources  and  improve 
performance  &  SA 

Payoffs 

•  Monitoring  and  diagnostic  tools  for  dynamic  assessment,  management,  and  mitigation  of 
teams,  improving  performance  and  SA 

•  Provides  critical  metrics  for  understanding  human-unmanned  systems 

-  Such  systems  operate  differently  than  human  teams  in  many  respects  (e.g.,  issues  of  trust, 
complacency,  communication,  etc.) 

-  TRACE  provides  additional/novel  approaches  to  understand  teams,  team  processes 

•  A  diverse  suite  of  validated  team  process  metrics,  allowing  more  accurate  appraisal  of  team 
effectiveness 

-  Allows  us  to  treat  teams  as  emergent  systems,  not  simply  collections  of  individuals 
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Cerebral  Hemodynamics 


•  Transcranial  Doppler  Sonography 
(TCD) 

-  Utilizes  ultrasound  signals  to  monitor 
intracranial  arteries 

-  When  a  particular  area  of  the  brain 
becomes  metabolically  active,  by¬ 
products  of  this  activity  will  increase 

•  This  results  in  increased  blood  flow  to  the 
region  to  remove  the  unwanted  by-products 

•  Near-Infrared  Spectroscopy  (NIRS) 

-  Utilizes  tissue  absorption  of  near- 
infrared  wavelengths  to  measure 
cortical  oxygen  saturation  levels  or 
regional  saturation  of  oxygen  (rS02) 
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Voice  Stress  Analysis 
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Fractal  analysis  of  human  inter-beat  interval  data 


Sensitivity  &  Diagnosticity  in 
Predicting  Team  Performance 


•  HMM  development  extracts  patterns  of  behavior 
from  large  corpora  of  training  data 

-  Future  prediction  is  based  on  statistical 
likelihoods  of  a  chain  of  behavior  derived  from 
patterns  learned  in  training 

-  Provide  a  novel  means  to  monitor  and  predict 
individual  and  team  performance 

-  Uncertain  if  predictive  accuracy  is  improved 
using  separate  HMMs  for  each  team  member, 
or  using  a  single  “team”  HMM 

-  HMM  prediction  under  different  levels  of  task 
demand 


•  EEG-based  measures  of  workload  are  particularly 
promising  (Gevins  &  Smith,  2003) 

-  Central  assumption  is  that  changes  in  brain 
activity  reflect  ongoing  mental  work  (Tsang  & 
Vidulich,  2006) 
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RHCPT  Team 


Research  Scientists 

-  Gregory  Funke,  Ph.D. 

-  Benjamin  Knott,  Ph.D. 

-  Lt  Connie  Ambrose 

-  Becky  Brown 

-  April  Courtice* 

-  Matthew  Funke* 

-  Maj  Chris  McClernon, 
Ph.D. 

-  April  Rose  Panganiban 

-  Sheldon  Russell* 


Software  Engineers 

-  Allen  Dukes 

-  Brent  Miller 

-  Jim  Hyson 

-  Matt  Middendorf 
Program  Managers 

-  Sam  Kuper 


*students 
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RHCPT  Collaborators 


•  AFRL 

-  RHCPA  -  RHXS 

-  RHCB  -  RHCI 

-  RHA  -  RISA 

•  International  Partners 

-  DSTL  &  QinetiQ,  UK 

-  DSTO,  AUS 

•  Academic  Partners 

-  Massachusetts  Institute  of  Technology  (MIT),  Humans  and  Automation 
Laboratory  (HAL) 

-  University  of  Cincinnati 

-  Wright  State  University 

-  West  Point 

-  University  of  Central  Florida 

•  Industry 

-  Boeing 


Miir 


Cincinnati 


UNIVERSITY  OF 


WRIGHT  STATE 


UNIVERSITY  OF  CENTRAL  FLORIDA 


Collaborative  Tools  for  C2 


Questions? 
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RHCPT  Spaces  and  Projects 


•  CTT  Lab 

-  SDO  Program  -  Sam  Kuper 

-  Dynamic  Cyber  Security  -  Janet 
Peasant 

-  MATRIX  Experiment 

•  DRAW  &  UK  PA -Allen  Dukes 

•  WCAS  &  MMC  -  Brent  Miller 

•  SPO  Chat  Study  -  April  Courtice 

•  Nonlinear  Analysis  -  Sheldon 
Russell 

•  Workload  Lab 

-  Voice  Stress  Analysis 

•  Algorithms  and  Metrics  -  Chris 
McClernon,  Matt  Middendorf 

•  Nonverbal  Voice  Stress  Analysis  - 
Mike  Harter 

-  BioRadios  &  EEG  -  Becky  Brown, 
April  Rose  Panganiban 


•  DART  Lab 

-  Change  Blindness  Research  -  April 
Rose  Panganiban,  Becky  Brown 

•  BMC2  Lab 

-  Transcranial  Doppler  Sonography 
Research 

•  TCD  and  Vigilance  Research  - 
Matt  Funke 

•  Removal  of  Voice-Related  Artifacts 
from  TCD  Recording  -  Connie 
Ambrose 

-  Overview  of  TRACE  Research 

•  FaceLab  Demo  -  Allen  Dukes 

•  TRACE  Workload  Scale  and 
Exchange  Interfaces  -  Jim  Hyson 
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